REMARKS 



In view of the foHowing reasoning for aHowance, the applicants iiereby 
respectfully request further examination and reconsideration of the subject 
application. 

A. interview Summary. 

The following paragraph summarizes the interview between Examiner Shashi 
Becker and the applicant's representative Katrina Lyon on 16 July 2007, to discuss 
the applicant's response to the Final Office Action of May 9, 2007. The Examiner 
clarified her rejections saying that she relied on Dayton to teach an image stack 
where layers of an image are the images of the image stack. The applicant's 
representative stated that the claim language identified an image stack as a stack of 
original images taken from the same point of view, which is different than the layers 
of an image. Since an agreement couid not be reached the applicant's 
representative agreed to darify the daim language to differentiate the image stack 
more distinctly from a layered image. As a result, the claims have be amended to 
define the image stack as a stack of non-layered, separate original images , each 
original image taken at a distinct separate time from the same point of view. It is 
believed that this amendment clearly distinguishes the claims from the cited art. 

The following paragraphs address the rejections cited in the Final Office 

Action. 

B. Claim Rejection Under 35 USC 112 

Claim 36 was rejected under 35 USC 112, first paragraph, for failing to 
comply with the written description requirement. Claim 36 is a dependent daim of 
Claim 33 "wherein scaling the source image or the composite image scales paint 
brush function". The applicants contend that scaling of obiects in a graphical user 
interface is well known to people with ordinary skill in the art, so that no detailed 
information should be required on this point. However, since so requested by the 



Examiner, the Appiicants with the response to the Final Office Action have provided 
evidence, downloaded from the popular Wikipedia website, that scaling of objects is 
well known. 

C. The 35 USC 103 Rejection of Claims 1.4. 7-13. 16. 18-23. 28. 29 and 30- 
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In the aforementioned Final Office Action, Claims 1,4, 7-13, 16, 18-23, 28, 29 
and 30-37 were rejected under 35 USC 103(a) as being unpatentable over Dayton in 
view of Hsu. U.S. Patent No. 6.078,701. The Examiner contended that Dayton teaches 
all the elements of the applicants claims but does not teach the applicant's image stack. 
The Examiner contended, however, that Hsu teaches this feature. The applicants 
respectfully disagree with this contention of obviousness. 

in order to deem the applicants' claimed invention unpatentable under 35 USC 
103, a prima facie showing of obviousness must be made. To make a prima facie 
showing of obviousness, all of the claimed elements of an applicants' invention must be 
considered, especially when they are missing from the prior art. If a claimed element is 
not taught in the prior art and has advantages not appreciated by the prior art:, then no 
prima facie case of obviousness exists. The Federal Circuit court has stated that it was 
error not to distinguish claims over a combination of prior art references where a 
material limitation in the claimed system and Its purpose was not taught therein {In Re 
Fine, 837 F.2d 107. 5 USPQ2d 1596 (Fed. Cir. 1988)). 

The applicants' claimed invention employs an "image stack" in easily 
combining individual images into an enhanced composite image. An image stack is 
a set of separate non-layered identically-sized registered images (e.g., the same 
pixel in each image represents more or less the same thing and inherently they are 
taken from the same point of view) that may originate from any stationary still or 
video camera, each image being taken at a distinct time. One way to envision an 
image stack is as a three dimensional (3D) coHection of pixels defined by a set of 
images (or a short video), in the 3D pixel set, the normal Xand V dimensions define 
the coordinates of a single image. The Z (or time) dimension defines which image in 



the stack (or what point in time in a video). A span of pixels is the set of ail pixels at 
some (X, Y) location in all images of the image stack. Filters may be applied to the 
3D image stack, or a portion thereof, to create one or more new 2D intermediate 
images. A filter is a function that operates on the 3D image stack to create a 2D 

image. An intermediate image is one created by running a filter on the image stack, 
where the image stack is a stack of non-iavered, separate origlnai images, each 
original image taken at a distinct separate time from the same point of view, (see 
Summary) 

in contrast Dayton teaches a technique for applying filters to a single image. 
Nowhere does Dayton teach an image stack "comprising a stack of no n- layered, 
separate original images, each original image taken at a distinct separate time from 
the same point of view, wherein the pixel position of each original image in the 
image stack is defined in a three dimensional coordinate system, and wherein two 
dimensions describe the dimensions of each image in the image stack, and the third 
dimension describes the time an image was captured..." 

Hsu teaches a method and apparatus that determines the topology of a 
sequence of images and then giobaliy aligns the images with respect to each 
image's nesghbonng images. The apparatus includes a topology determination 
module, a local coarse registration module, a iocai fine registration module, a global 
consistency module, and a color matching/blending module. To accurately render a 
mosaic image from a sequence of images the topology determination and global 
alignment processes are iterated to progressively produce accurately aligned 
images. The apparatus efficiently and accurately combines a plurality of source 
images into a seamless panoramic mosaic representation of a scene, of arbitrarily 
wide spatial and angular extent, regardless of the order of image acquisition. 
However, Hsu technique performs image processing by stitching together images to 
create a single tiled image. Hsu's technique uses different points of camera view to 
create this mosaic in cases where a camera is used to travel in a straight line and 
capture images of a scene (column 18, lines 9-54, column 16, lines 44-67). 
Therefore, Hsu also does not teach an image stack "comprising a stack of 
original images a stack of non-layered, separate original images, each original 
image taken at a distinct separate time from the same point of view, wherein 



the pixel position of each original image in the image stack is defined in a 
three dimensional coordinate system, and wherein two dimensions describe 
the dimensions of each image in the image stack, and the third dimension 
describes the time an image was captured..." 



Granted, the Examiner states that Hsu teaches the image stack made of 
images taken from the same point of view at column 18, lines 30-46 and at column 
line 44-column 17 line 9. However, If one looks at these paragraphs, as well as the 
paragraphs immediately before and after these paragraphs it is clear that Hsu 
teaches an image stack created of images taken from multiple points of view . For 
example, column 16 lines 44-46 state, 



"The invention as described above creates mosaics of scenes from 
two-dimensional imagery. Computation of the three-dimensional structure and 
its representation has not been explicitly dealt with in the foregoing 
discussion. The following describes an illustrative process for extending the 
invention to include generation of three-dimensional mosaics representing a 
three-dimensional scene. Prior techniques for processing and utilizing 
three-dimensional representations of scenes are disclosed in U.S. 
patent application Ser, No. 08/499,934 filed Jul. 10, 1995, which 
discloses view-based three-dimensional representations from a local 
collection of viewpoints. The present invention can be used to extend 
these techniques to create three-dimensional spherical mosaic 
representations of extended scenes. 

I n order to represent the three-dimensional information of a scene from 
a cjfven viewpoint, a twoHjimg spherical mosaic is created from 

thg gjyeri-YjQwpoi nt and then from a fewjtiore yiew poin ts (typicatly one 
more but may be more) by. changing the locati on of the eamera, This 
process creates extended views of the scene, one each from the chosen 
amera positions. One main advantage of creating a spherlca! mosaic 
representation from every viewpoint is that each such viewpoint provides a 
much wider field of view than just a single image from that viewpoint." 
(emphasis added) 

Likewise, column 18 lines 9-29 state, 

"When the camera is moved through an environment, in general it 

changes orientatfon (rotates) and changes location (translates). In this 
situation, a mosaic of the scene cannot be created by a coordinate system 
located at/around one center of projection. The present invention can be used 
to solve this problem for some specific situations. 
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The key idea that is exploited here for creating mosaics with arbitrary 
motions of the camera is that no explicit depth reconstruction is 
required. The mosaics are created by assembling together closeiy spaced 

registered strips of images. 

In order to describe the basic ideas of this implementation of the 
invention, the invention shall first be described in the context of the 
simplest case for three-dimensional mosaics: a camera moving in a 
straight line and looking at right angles to the direction of motion. The 
image planes are all coplanar for this case. Assuming continuous images and 
continuous motion, the central vertical line of each image is seen only in that 
image and hence by stacking together these central lines, a three- 
dimensional mosaic of arbitrary extent may be created. 

Another way to create such a mosaic is to stack each of the images in 
an xyt cube where xy is the spatial image coordinate system and t is the time 
dimension. In the continuous case (or the dense sampling case), a slice 
through this cube along the y=y.sub.c (where y.sub.c is the y co-ordinate of 
the center of the image) plane creates the required mosaic. 

In the more realistic situation of finite sampling of the frames In time, it 
has been shown in the art that any arbitrary view of the scene between 
any two views may be created by linear Interpolation of the flow vectors 
between two consecutive images. This implies that even when the 
captured frames are coarsely sampled in time, an arbitrarily dense time 
sampling may be created by view interpolation. Once such a sampling is 
available, the creation of the mosaic is trivial as discussed above. Note that 
the densely sampled complete image frames need not be created but only 
the central slit of each frame." (emphasis added) 

Neither Dayton nor Hsu teach the applicants' claimed image stack of non- 
lavered, se parate original images , each original image taken at a distinct separate 
time from the same point of view, which may be used for a variety of applications 
such as, for example, creating high dynamic range images, combining images 
captured under different lighting conditions, removing objects from images, and 
combining images captured at multiple points in time or with different focal lengths. 



Additionally, the Dayton and Hsu references do not teach the advantageous 
features of the applicants' claimed invention such as being able to create a variety of 
special effects using the image stack. Accordingly, no prima facie case of 
obviousness has been established in accordance with the holding of In Re Fine. 
This lack of prima facie showing of obviousness means that the rejected claims are 
patentable under 35 USC 103 over Dayton in view of Hsu. As such, it is respectfully 
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requested that Ciaims 1, 4, 7-13, 16, 18-23, 28, 29 and 30-37 be allowed based on 
the following exemplary claim language: 



" A computer-impiemented process for creating a composite image, 
comprising using a computer to perform the following process actions; 

inputting an image stack comprising a stack of non-fayered, 
separate original images, each oriMnaf image taken at a distjnct 
separate time from the same point of view, wherein the pixel position of 
each original image in the image stack is defined in a three dimensional 
coordinate system, and wherein two dimensions describe the 
dimensions of each image in the image stack, and the third dimension 
describes the time an image was captured; 

applying one or more filters to the image stack to create one or more 
new intermediate images; 

selecting one of the original images in the image stack or an 
intermediate image as a source image; and 

selecting pixels from the source image to be added to a composite 
image to create a final composite image." 



D. The 35 USC 103 Rejection of Claims 2. 5. and 6. 

In the Final Office Action, Claims 2, 5 and 6 were rejected under 35 USC 103(a) 
as being unpatentable over Dayton and Hsu (as discussed above) in further view of 
Wise (U.S. Patent No. 6,130,676), herein after Wise. The Examiner contended that 
though Dayton and Hsu do not teach various features of these claims, Wise teaches 
these features. The applicants respectfully disagree with this contention of 
obviousness. 

The applicants' claimed invention employs an "image stack comprising a 
stack of non-lavered. separate original images , each oriqinai image taken at a 
d isti n ct sepa ra te time fro m the same point of view, wherein the pixel position of each 
original image in the image stack is defined in a three dimensional coordinate 
system, and wherein two dimensions describe the dimensions of each image in the 
image stack, and the third dimension describes the time an image was captured in 
easily combining individual images into an enhanced composite image. One way to 
envision an image stack is as a three dimensional (3D) collection of pixels defined 
by a set of images (or a short video). In the 3D pixel set, the normal X and Y 
dimensions define the coordinates of a single image. The Z (or time) dimension 
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defines which image in the stack (or what point in time in a video). Inherantly the 
images are taken from the same point of view because the pixels in each image 
more or less represent the same thing (see Summary). 

As discussed above, neither Dayton nor Hsu teach an image stack wherein 
images are taken from the same point of view. Wise also does not teach this 

feature. Additionally, the Dayton, Hsu and Wise references do not teach the 
advantageous features of the applicants' claimed invention such as being able to 
create a variety of special effects using the image stack. Accordingly, no prima fade 
case of obviousness has been established in accordance with the holding of In Re 
Fine. This lack of prima facie showing of obviousness means that the rejected 
claims are patentable under 35 (JSC 103 over Dayton in view of Hsu and Wise. As 
such, it is respectfully requested that Claims 2, 5 and 6 be allowed based on the 
aforementioned claim language. 

E. The 35 USC 103 Refection of Claims 3 and 17. 

In the Final Office Action, Claims 3 and 17 were rejected under 36 USC 103(a) 
as being unpatentable over Dayton and Hsu (as discussed above) in further view of 
Joidon (U.S. Patent No. 5,493,419), herein after Joidon. The Examiner contended that 
though Dayton and Hsu do not teach applying a slice filter, Joidon teaches this feature. 
The applicants respectfully disagree with this contention of obviousness. 

Neither Dayton nor Hsu teach the applicants' cfaimed image stack, 
comprising a stack of non-layered, separate original images , each original image 
taken at a distinct separate time from the same point of view, wherein the pixel 
position of each original image in the image stack is defined in a three dimensional 
coordinate system, and wherein two dimensions describe the dimensions of each 
image in the image stack, and the third dimension describes the time an image was 
captured wherein images are taken from the same point of view, which may be used 
for a variety of applications such as, for example, creating high dynamic range 
images, combining images captured under different lighting conditions, removing 



objects from images, and combining images captured at multiple points in time or 
with different focal lengths. 

Joidon also does not teach the applicants' claimed image stack wherein non- 
layered separate images are taken from the same point of view at distinctly different 
times. 

Additionally, the Dayton, Hsu and Joidon references do not teach the 
advantageous features of the applicants' ciaimed invention such as being able to 
create a variety of speciai effects using the image stack. Accordingly, no prima facie 
case of obviousness has been established in accordance vvsih the holding of In Re 
Fine. This lack of prima fade showing of obviousness means that the rejected 
claims are patentable under 35 USC 103 over Dayton in view of Hsu and Joidon, As 
such, it is respectfully requested that Claims 3 and 17 be allowed based on the 
aforementioned claim language. 

F. The 35 USC 103 Rejection of Claims 14. 26 and 27. 

In the Final Office Action, Claims 14, 26 and 27 were rejected under 35 USC 
1 03(a) as being unpatentable over Dayton and Hsu (as discussed above) in further view 
of Okamoto et at (U.S. Patent No. 5754618), herein after Okamoto. The Examiner 
contended that though Dayton and Hsu do not teach applying a surface filter that 
operates on a given surface through the image stack and a surface within the image 
stack that is user defined, Okamoto teaches this feature. The applicants respectfully 
disagree with this contention of obviousness. 

As mentioned previously, neither Dayton nor Hsu teach the applicants' 
claimed image stack, wherein images are taken from the same point of view, and 
which may be used for a variety of applications such as, for example, creating high 
dynamic range images, combining images captured under different lighting 
conditions, removing objects from images, and combining images captured at 
multiple points in time or with different focal lengths. 



18 



Okamoto also does not teach the applicant's claimed image stack of original 
non-layered images taken at different times wherein the third dimension represents 
time and wherein images are taken from the same point of view. 

Additionally, the Dayton, Hsu and Okamoto references do not teach the 
advantageous features of the applicants' claimed invention such as being able to 
create a variety of special effects using the image stack wherein images are taken 
from the same point of view. Accordingly, no prima facie case of obviousness has 

been established in accordance with the holding of In Re Fine. This lack of prima 
facie showing of obviousness means that the rejected claims are patentable under 
35 use 103 over Dayton in view of Hsu and Okamoto, As such, it Is respectfully 
requested that Claims 14, 26 and 27 be allowed based on the aforementioned claim 
language. 

G. The 35 USC 103 Relection of Claims 15. 24 and 25. 

In the Final Office Action, Claims 15, 24 and 25 were rejected under 35 USC 
103(a) as being unpatentable over Dayton and Hsu (as discussed above) in further view 
of Chuang et al., "Video Matting of Complex Scenes", herein after Chuang. The 
Examiner contended that though Dayton and Hsu do not teach applying a mat filter that 
produces a mat of a given portion of the image stack, Chuang teaches this feature. 
The applicants respectfully disagree with this contention of obviousness. 

As discussed previously neither Dayton nor Hsu teach the applicants' claimed 
image stack wherein the images are a stack of non-layered, separate original 
images , each original image taken at a distinct separate time from the same point of 
view, wherein the pixel position of each original image in the image stack is defined 
in a three dimensional coordinate system, and wherein two dimensions describe the 
dimensions of each image in the image stack, and the third dimension describes the 
time an image was captured 

Chuang also does not teach the applicant's claimed image stack wherein 
non-layered images are taken from the same point of view at distinct times. 



Addittonaliy, the Dayton, Hsu and Chuang references do not teach the 
advantageous features of the appiicants' claimed invention such as being able to 
create a variety of special effects using the image stack. Accordingly, no prima facie 
case of obviousness has been established in accordance with the holding of In Re 
Fine. This lack of prima facie showing of obviousness means that the rejected 
claims are patentable under 35 USC 103 over Dayton in view of Hsu and Chuang. 
As such, it is respectfully requested that Claims 15, 24 and 25 be allowed based on 
the aforementioned claim language. 

H. The 35 USC 103 Rejection of Claim 38. 

In the Final Office Action. Claim 38 was rejected under 35 USC 103(a) as being 
unpatentable over Dayton and Hsu (as discussed above) in further view of Funayama 
et al. (U.S. Patent No. 6,389,155), herein after Funayama. The Examiner contended 
that though Dayton and Hsu do not teach a paint brush function that transfers all pixels 
associated with a face from a source image to a composite image whent said paint 
brush function is used to select a portion of a face, Funayama teaches these features. 
The applicants respectfully disagree with this contention of obviousness. 

Neither Dayton nor Hsu teach the applicants' claimed image stack 
comprising a stack of non-layered, separate original images , each original image 
taken at a disiinct separate time from the same point of view, wherein the pixel 
position of each original image in the image stack is defined in a three dimensional 
coordinate system, and wherein two dimensions describe the dimensions of each 
image in the image stack, and the third dimension describes the time an image was 
capturedand which may be used for a variety of applications such as, for example, 
creating high dynamic range images, combining images captured under different 
lighting conditions, removing objects from images, and combining images captured 
at multiple points in time or with different focal lengths. 

Funyama also does not teach the applicant's claimed image stack of non- 
layered images taken at distinct times and from the same point of view. 
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Additionaily, the Dayton, Hsu and Funayama references do not teach the 
advantageous features of the applicants' claimed invention such as being able to 
create a variety of special effects using the image stack. Accordingly, no prima facie 
case of obviousness has been established in accordance with the holding of In Re 
Fine. This lack of prima facie showing of obviousness means that the rejected 
claims are patentable under 35 (JSC 103 over Dayton in view of Hsu and Funayama. 
As such, it is respectfully requested that Claim 38 be allowed based on the 
aforementioned claim language. 

in summary, it is believed that the claims 1-38 are in condition for ailowance. 
Allowance of these claims at an early date is courteously solicited. 
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